[SPARK-33717][LAUNCHER] deprecate spark.launcher.childConectionTimeout #30679

jsoref · 2020-12-09T01:28:47Z

What changes were proposed in this pull request?

Deprecating spark.launcher.childConectionTimeout in favor of spark.launcher.childConnectionTimeout

Why are the changes needed?

How was this patch tested?

No testing. Not even compiled

HyukjinKwon · 2020-12-09T01:40:48Z

For API deprecation, we should file a JIRA, @jsoref. Can you file a JIRA at https://issues.apache.org/jira/projects/SPARK/issues? Also it's best to keep the PR titles properly. See also https://spark.apache.org/contributing.html

github-actions · 2021-03-19T02:56:34Z

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

jsoref · 2021-03-19T03:05:06Z

@HyukjinKwon: I filed https://issues.apache.org/jira/browse/SPARK-33717...

@srowen is there something I need to do to push this along?

srowen · 2021-03-19T17:56:01Z

Jenkins test this please

SparkQA · 2021-03-19T18:31:30Z

Test build #136266 has finished for PR 30679 at commit 5650822.

This patch fails Java style tests.
This patch merges cleanly.
This patch adds no public classes.

jsoref · 2021-03-21T01:32:14Z

@srowen: sigh. can you provide the error output? (I can't find it from the bot's link) (and also the other bot killed this PR)

srowen · 2021-03-21T01:34:47Z

See the output linked above. It seems to be:

[error] Checkstyle error found in /home/jenkins/workspace/SparkPullRequestBuilder/launcher/src/main/java/org/apache/spark/launcher/LauncherServer.java:266: 'if' construct must use '{}'s.
[error] Checkstyle error found in /home/jenkins/workspace/SparkPullRequestBuilder/launcher/src/main/java/org/apache/spark/launcher/LauncherServer.java:272: Line is longer than 100 characters (found 104).
[error] Checkstyle error found in /home/jenkins/workspace/SparkPullRequestBuilder/launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java:99: Line is longer than 100 characters (found 106).
[error] 3 issue(s) found in Checkstyle report: /home/jenkins/workspace/SparkPullRequestBuilder/launcher/target/checkstyle-output.xml
[error] com.etsy.sbt.checkstyle.NoExitException

SparkQA · 2021-03-21T02:49:17Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40875/

SparkQA · 2021-03-21T02:57:57Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40875/

SparkQA · 2021-03-21T03:53:43Z

Test build #136293 has finished for PR 30679 at commit e1db9f4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

jsoref · 2021-03-22T00:54:46Z

Can someone please whack the bot? This is insane.

srowen · 2021-03-22T01:54:12Z

I cleared the 'stale' tag - that was the issue

SparkQA · 2021-03-22T02:09:40Z

Test build #136319 has started for PR 30679 at commit e1db9f4.

SparkQA · 2021-03-22T03:11:36Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40901/

SparkQA · 2021-03-22T03:24:33Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40901/

srowen · 2021-03-22T14:45:59Z

Jenkins retest this please

SparkQA · 2021-03-22T16:17:28Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40937/

SparkQA · 2021-03-22T16:23:14Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40937/

SparkQA · 2021-03-22T18:49:17Z

Test build #136353 has finished for PR 30679 at commit e1db9f4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

One more thing would finish this out. Let's add a note in docs/core-migration-guide.md, under the 3.1 -> 3.2 heading, like:

In Spark 3.2, spark.launcher.childConectionTimeout is deprecated (typo) though still works. Use spark.launcher.childConnectionTimeout instead.

launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java

in favor of spark.launcher.childConnectionTimeout Signed-off-by: Josh Soref <[email protected]>

jsoref · 2021-03-23T14:56:07Z

Unrelated, markdownlint (triggered automatically in VSCode) suggests changing - to * for top level bullets: https://github.com/DavidAnson/markdownlint/blob/v0.23.1/doc/Rules.md#md004

https://issues.apache.org/jira/browse/SPARK-34838

srowen · 2021-03-23T15:08:19Z

Probably so - we can think about that separately if it's a problem.

srowen · 2021-03-23T15:08:25Z

Jenkins retest this please

SparkQA · 2021-03-23T18:30:01Z

Test build #136415 has finished for PR 30679 at commit 056d432.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-03-23T19:52:39Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40999/

srowen · 2021-03-23T22:58:49Z

Jenkins retest this please

SparkQA · 2021-03-24T01:12:26Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41010/

SparkQA · 2021-03-24T02:05:30Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41010/

srowen · 2021-03-24T13:48:45Z

Jenkins retest this please

SparkQA · 2021-03-24T14:16:45Z

Test build #136468 has started for PR 30679 at commit 056d432.

SparkQA · 2021-03-24T19:39:43Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41052/

SparkQA · 2021-03-24T20:16:23Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41052/

srowen · 2021-03-26T17:29:32Z

Jenkins retest this please

SparkQA · 2021-03-26T18:20:41Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41158/

SparkQA · 2021-03-26T18:28:12Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41158/

SparkQA · 2021-03-26T20:04:21Z

Test build #136574 has finished for PR 30679 at commit 056d432.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2021-03-26T20:54:00Z

Merged to master

jsoref changed the title ~~spelling: [API] deprecate spark.launcher.childConectionTimeout~~ [SPARK-33717][LAUNCHER] deprecate spark.launcher.childConectionTimeout Dec 9, 2020

jsoref mentioned this pull request Dec 11, 2020

Spelling #30323

Closed

maropu mentioned this pull request Feb 28, 2021

[SPARK-34566][LAUNCHER] Fix configuration typo of spark. launcher.childConnectionTimeout #31678

Closed

github-actions bot added the Stale label Mar 19, 2021

github-actions bot closed this Mar 20, 2021

srowen reopened this Mar 21, 2021

jsoref force-pushed the spelling-connection branch from 5650822 to e1db9f4 Compare March 21, 2021 01:48

github-actions bot closed this Mar 22, 2021

srowen removed the Stale label Mar 22, 2021

srowen reopened this Mar 22, 2021

srowen reviewed Mar 23, 2021

View reviewed changes

launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java Outdated Show resolved Hide resolved

spelling: [API] deprecate spark.launcher.childConectionTimeout

056d432

in favor of spark.launcher.childConnectionTimeout Signed-off-by: Josh Soref <[email protected]>

jsoref force-pushed the spelling-connection branch from e1db9f4 to 056d432 Compare March 23, 2021 14:52

github-actions bot added the DOCS label Mar 23, 2021

srowen closed this in d58587b Mar 26, 2021

jsoref deleted the spelling-connection branch March 30, 2021 01:16

[SPARK-33717][LAUNCHER] deprecate spark.launcher.childConectionTimeout #30679

[SPARK-33717][LAUNCHER] deprecate spark.launcher.childConectionTimeout #30679

Uh oh!

Conversation

jsoref commented Dec 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

How was this patch tested?

Uh oh!

HyukjinKwon commented Dec 9, 2020

Uh oh!

github-actions bot commented Mar 19, 2021

Uh oh!

jsoref commented Mar 19, 2021

Uh oh!

srowen commented Mar 19, 2021

Uh oh!

SparkQA commented Mar 19, 2021

Uh oh!

jsoref commented Mar 21, 2021

Uh oh!

srowen commented Mar 21, 2021

Uh oh!

SparkQA commented Mar 21, 2021

Uh oh!

SparkQA commented Mar 21, 2021

Uh oh!

SparkQA commented Mar 21, 2021

Uh oh!

jsoref commented Mar 22, 2021

Uh oh!

srowen commented Mar 22, 2021

Uh oh!

SparkQA commented Mar 22, 2021

Uh oh!

SparkQA commented Mar 22, 2021

Uh oh!

SparkQA commented Mar 22, 2021

Uh oh!

srowen commented Mar 22, 2021

Uh oh!

SparkQA commented Mar 22, 2021

Uh oh!

SparkQA commented Mar 22, 2021

Uh oh!

SparkQA commented Mar 22, 2021

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jsoref commented Mar 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

srowen commented Mar 23, 2021

Uh oh!

srowen commented Mar 23, 2021

Uh oh!

SparkQA commented Mar 23, 2021

Uh oh!

SparkQA commented Mar 23, 2021

Uh oh!

srowen commented Mar 23, 2021

Uh oh!

SparkQA commented Mar 24, 2021

Uh oh!

SparkQA commented Mar 24, 2021

Uh oh!

srowen commented Mar 24, 2021

Uh oh!

SparkQA commented Mar 24, 2021

Uh oh!

SparkQA commented Mar 24, 2021

Uh oh!

SparkQA commented Mar 24, 2021

Uh oh!

srowen commented Mar 26, 2021

Uh oh!

SparkQA commented Mar 26, 2021

jsoref commented Dec 9, 2020 •

edited

Loading

jsoref commented Mar 23, 2021 •

edited

Loading